Image pickup apparatus that shoots moving image for predetermined time period at the time of shooting still image, control method for the image pickup apparatus, and storage medium

ABSTRACT

An image pickup apparatus which is capable of shooting a plurality of still images as instructed within a predetermined time period without a plurality of moving image encoders and a large-capacity memory. When a second still image shooting instruction has been received within a predetermined time period since a first still image shooting instruction was received, a moving image picked up before lapse of the predetermined time period after the second still image shooting instruction from a moving image shooting starting timing corresponding to the first still image shooting instruction is recorded as a sequence of moving images.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image pickup apparatus such as adigital camera, and in particular to an image pickup apparatus thatshoots a moving image for a predetermined time period before or aftershooting a still image, a control method for the image pickup apparatus,and a storage medium.

Description of the Related Art

Some image pickup apparatuses such as digital camera shoot a movingimage for a predetermined time period before and after shooting a stillimage.

For example, there has been proposed an image pickup apparatus whichstarts recording a moving image at first releasing by pressing a releasebutton halfway down, shooting a still image at second releasing bypressing the release button all the way down, and finishes shooting themoving image upon lapse of a predetermined time period after shooting ofthe still image (Japanese Laid-Open Patent Publication (Kokai) No.2001-78137).

The art described in Japanese Laid-Open Patent Publication (Kokai) No.2001-78137 above has problems described hereafter. They will now bedescribed with reference to FIG. 7. In FIG. 7, T1 designates a time atwhich a first shooting instruction is issued, and T2 designates a timeat which a second shooting instruction is issued. MP1 designates amoving image shooting time period corresponding to a predetermined timeperiod before and after T1, and MP2 designates a moving image shootingtime period corresponding to a predetermined time period before andafter T2.

As shown in FIG. 7, when both a moving image for a predetermined timeperiod before and after shooting of a first still image (MP1) and amoving image for a predetermined time period before and after shootingof a second still image (MP2) are to be recorded, there is anoverlapping time period, and hence the two moving images must be encodedin parallel at the same time.

In order to carry out a plurality of moving image encoding processes inparallel, a plurality of moving image encoders and a large-capacitymemory for temporarily recording moving images including those for anoverlapping moving image shooting timer period are necessary. This maycause an increase in the size of the image pickup apparatus and anincrease in costs.

If a plurality of encoders and a large-capacity memory are not providedwith an image pickup apparatus, a plurality of moving image encodingprocesses cannot be performed in parallel. In this case, it isconceivable that when a moving image encoding process is underway aftershooting of a first still image, an instruction to shoot a second stillimage is not received. In this case, however, the chance of shooting ina predetermined time period after shooting of the first still image ismissed.

SUMMARY OF THE INVENTION

The present invention provides an image pickup apparatus which shoots amoving image for a predetermined time period before or after shooting astill image and is capable of shooting a plurality of still images asinstructed within a predetermined time period without the need to have aplurality of moving image encoders and a large-capacity memory, acontrol method for the image pickup apparatus, and a storage medium.

Accordingly, the present invention provides an image pickup apparatuscomprising a receiving unit configured to receive a still image shootinginstruction, an image pickup unit, and a control unit configured tocontrol to record, in a recording medium, a still image picked up by theimage pickup unit in response to a still image shooting instructionreceived by the receiving unit, and a moving image picked up beforelapse of a predetermined time period after the still image shootinginstruction, and when the receiving unit has received a second stillimage shooting instruction within the predetermined time period sincethe receiving unit received a first still image shooting instruction,control to record, in the recording medium, a moving image picked upbefore lapse of the predetermined time period after the second stillimage shooting instruction from a moving image shooting starting timingcorresponding to the first still image shooting instruction as asequence of moving images.

According to the present invention, the image pickup apparatus whichshoots a moving image for a predetermined time period before or aftershooting a still image is capable of shooting a plurality of stillimages as instructed within a predetermined time period without the needto have a plurality of moving image encoders and a large-capacitymemory.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an exemplary arrangement of a digitalcamera that is an exemplary embodiment of an image pickup apparatusaccording to the present invention.

FIG. 2 is a block diagram showing an exemplary arrangement of an imageprocessing circuit.

FIG. 3 is a flowchart useful in explaining a process in a stillimage-moving image shooting mode.

FIG. 4A is a flowchart showing the procedure of a process in a recordingmode 1 in step S315 in FIG. 3, and FIG. 4B is a flowchart showing theprocedure of a process in a recording mode 2 in step S316 in FIG. 3.

FIGS. 5A and 5B are schematic diagrams useful in explaining an exemplarystill image temporary recording process in step S306 in FIG. 3.

FIGS. 6A to 6C are schematic diagrams useful in explaining how stillimage data and moving image frame data are recorded in the recordingmode 1 and the recording mode 2.

FIG. 7 is a schematic diagram useful in explaining problems of a priorart.

DESCRIPTION OF THE EMBODIMENTS

Hereafter, an exemplary embodiment of the present invention will bedescribed with reference to the drawings.

FIG. 1 is a block diagram showing an exemplary arrangement of a digitalcamera that is an exemplary embodiment of an image pickup apparatusaccording to the present invention.

As shown in FIG. 1, the digital camera 100 according to the presentembodiment has taking lenses 103 including a zoom lens, a focus lens,and so forth, and a shutter 101 having a diaphragm function, and abundle of light rays from a subject passes through the taking lenses 103and the shutter to form a subject image on an image pickup unit 22. Theimage pickup unit 22, which is comprised of an image pickup device suchas a CCD sensor or a CMOS sensor, performs photoelectric conversion ofthe formed subject image and outputs an analog image signal obtained asa result to an A/D converter 23. The A/D converter 23 converts theanalog image signal output from the image pickup unit 22 into a digitalimage signal. A barrier 102 covers an image pickup system including thetaking lenses 103 to protect the taking lenses 103, the shutter 101, andthe image pickup unit 22.

An image processing circuit 24 performs a variety of image processing ondata from the A/D converter 23 or data from a memory control circuit 15.Data from the A/D converter 23 is written into a memory 32 through theimage processing circuit 24 and the memory control circuit or throughthe memory control circuit 15. The memory 32, which corresponds to anexemplary storage of the present invention, stores image data convertedinto digital data by the A/D converter 23 and image data to be displayedon a display unit 28. The memory 32 has a recording capacity sufficientto store a predetermined number of still images and store moving imagesand sound for a predetermined time period. The memory 32 also has amemory for displaying images (video memory).

A D/A converter 13 converts image data for display stored in the memory32 into an analog signal and supplies the analog signal to the displayunit 28. As a result, the image data for display written in the memory32 is displayed on the display unit 28 by way of the D/A converter 13.The display unit 28, which is comprised of an LCD or the like, displaysan image corresponding to an analog image signal from the D/A converter13. Once digital image signals have been obtained by A/D conversion bythe A/D converter 32 and accumulated in the memory 32, they areconverted into an analog signal by the D/A converter 13 and successivelytransferred to and displayed on the display unit 28, so that the displayunit 28 acts as an electronic viewfinder and displays through-the-lensimages.

A nonvolatile memory 56 is an electrically erasable programmable memorysuch as a ROM. Constants for operation of a system control circuit 50,programs, and so forth are recorded in the nonvolatile memory 56. Thesystem control circuit 50 is responsible for controlling the overalldigital camera 100. The system control circuit 50 also controls displayby controlling the memory 32, the D/A converter 13, the display unit 28,and so forth. A RAM is used as a system memory 52. The system memory 52expands constants for operation of the system control circuit 50,variables, programs read out from the nonvolatile memory 56, and soforth.

A system timer 53 is a time measurement unit that measures elapsed timefor use in a variety of control and measures elapsed time on a built-inclock. A mode selector switch 60 switches the operating mode of thesystem control circuit 50 among a still image shooting mode, a movingimage shooting mode, a reproducing mode, and so forth. Examples of modesincluded in the still image shooting mode are an auto shooting mode, anauto scene determination mode, a manual mode, various scene modes withshooting settings configured according to shooting scenes, a program AEmode, a still image-moving image shooting mode, and a custom mode. Here,in the still image-moving image shooting mode, a still image is shot inresponse to a shooting instruction, and moving images are shot for apredetermined time period before and after issuance of the shootinginstruction. It should be noted that after the operating mode isswitched once to the still image shooting mode using the mode selectorswitch 60, the operation mode may be switched to any of those modesincluded in the still image recording mode using another operatingmember. Likewise, a plurality of modes may be included in the movingimage shooting mode as well.

A release button 61 (shutter button) is an operating member (receivingmember) that receives a shooting instructing operation and a shootingpreparation instructing operation from a user. A release switch (SW1) 62is turned on in response to the release button 61 being pressed halfwaydown and instructs the system control circuit 50 to prepare forshooting. Based on an ON signal from the release switch (SW1) 62, thesystem control circuit 50 starts shooting preparations such as an AF(autofocus) process, an AE (auto exposure) process, an AWB (auto whitebalance) process, and an EF (pre-flash) process. A release switch (SW2)64 is turned on in response to the release button 51 being pressed allthe way down and instructs the system control circuit 50 to performshooting. Based on an ON signal from the release switch (SW2) 64, thesystem control circuit 50 starts a sequence of shooting operations fromreadout of a signal from the image pickup unit 22 to writing of imagedata into a recording medium 200.

An operating unit 70 is caused to act as various function buttons by,for example, selecting a variety of function icons, which are displayedon the display unit 28, to appropriately assign functions to situations.Examples of the function buttons include an end button, a back button,an image feed button, a jump button, a narrow-down button, and anattribute change button. For example, when a menu button is depressed, amenu screen on which various setting can be configured is displayed onthe display unit 28. A user is allowed to configure various settingsthrough intuition using the menu screen displayed on the display unit28, a four-way (up, down, right, and left) button, and a SET button.

A power switch 72 is for turning on and off the power to the digitalcamera 100. A power supply control unit 80 is comprised of a batterydetection circuit, a DC-DC converter, a switch circuit that switchesblocks to be energized, and so forth, and detects the presence orabsence of a battery mounted, a type of a battery, and a remainingcapacity of a battery. Also, based on a result of the detection and aninstruction from the system control unit 50, the power supply controlunit 80 controls the DC-DC converter and supplies required voltage tocomponents including the recording medium 200 over a required timeperiod.

A power supply unit 30 is comprised of a primary battery such as analkali battery or a lithium battery, a secondary battery such as a NiCdbattery, an NiMH battery, or an Li battery, an AC adapter, or the like.A recording medium I/F 18 is an interface to the recording medium 200such as a memory card or a hard disk. The recording medium 200 is arecording medium such as a memory card for recording shot images and iscomprised of a semiconductor memory, a magnetic disk, or the like.

A communication unit 54 is connected to external equipment by radio or awired cable and sends and receives video signals and audio signals. Thecommunication unit 54 is connectable to a wireless LAN (local areanetwork) and the Internet. The communication unit 54 is capable ofsending images picked up by the image pickup unit 22 (includingthrough-the-lens images) and images recorded on the recording medium 200to external equipment and also receiving image data and a variety ofother information from external equipment.

A posture sensing unit 55 senses a posture of the digital camera 100with respect to a direction of gravitational force. Based on a posturesensed by the posture sensing unit 55, it is possible to determinewhether an image picked up by the image pickup unit 22 is an imageobtained by holding the digital camera 100 in a landscape orientation oran image obtained by holding the digital camera 100 in a portraitorientation. The system control unit 50 is capable of adding orientationinformation corresponding to a posture sensed by the posture sensingunit 55 to an image file of an image picked up by the image pickup unit22 and rotating and recording images. An acceleration sensor, a gyrosenor, or the like may be used as the posture sensing unit 55.

Referring next to FIG. 2, a detailed description will be given of theimage processing circuit 24. FIG. 2 is a block diagram showing anexemplary arrangement of the image processing circuit 24.

Referring to FIG. 2, an image processing unit 201 carries out a resizingprocess such as predetermined pixel interpolation and reduction, a colorconversion process, and so forth. A computation process unit 202 carriesout a predetermined computation process using image data obtained byimage pickup, and based on a result of the computation, the systemcontrol unit 50 controls light exposure and distance measurement. As aresult, a TTL (through-the-lens) AF (autofocus) process, an AE (autoexposure) process, and an EF (pre-flash) process are carried out. Thecomputation process unit 202 also carries out a TTL AWB (auto whitebalance) process based on an obtained computation result.

Still image data is input to a still image encoding-decoding processunit 203, which in turn carries out an encoding process and generatesthumbnails. An encoding scheme is, for example, JPEG. Encoded stillimage data is recorded as a still image on the recording medium 200 viathe recording medium I/F 18. The still image encoding-decoding processunit 203 reads out thumbnail images and real images, which are recordedon the recording medium 200, via the recording medium I/F 18 and thememory control circuit 15 to carry out a decoding process.

A plurality of pieces of moving image frame data is input to a movingimage encoding-decoding process unit 204, which in turn carries out anencoding process. An encoding scheme is, for example, MPEG4 part-10: AVC(ISO/IEC 14496-10 sometime called H. 264). A plurality of pieces ofencoded moving image data is recorded as a moving image on the recordingmedium 200 via the recording medium I/F 18. The moving imageencoding-decoding process unit 204 reads out the moving image, which isrecorded on the recording medium 200, via the recording medium I/F 18and the memory control circuit 15 to carry out a decoding process.

Referring next to FIG. 3, a description will be given of the flow of aprocess in the still image-moving image shooting mode. It should benoted that processes in FIG. 3 are carried out by a CPU or the like ofthe system control circuit 50 in accordance with programs that arestored in the nonvolatile memory 56 and expanded into the system memory52.

Referring to FIG. 3, in step S300, when the still image-moving imageshooting mode is selected, the system control circuit 50 selects arecording mode 1 or a recording mode 2 according to an input from theoperating unit 70, and the process proceeds to step S301.

Here, the recording mode 1 is a shooting mode in which, for each of aplurality of shot still images, one still image and one moving imageshot within a predetermined time period before and after shooting of thestill image are recorded. The recording mode 2 is a shooting mode inwhich, for a single moving image continuously shot in a sequence, aplurality of still images shot while the moving image is being shot isrecorded. It should be noted that in the present embodiment, shootingmodes other than the recording mode 1 and the recording mode 2 areallowed to be selected, but here, only the recording mode 1 and therecording mode 2 are described for the simplification of explanation.

In the step S301, the system control circuit 50 initializes a durationof temporary recording and an SW2 flag, and the process proceeds to stepS302. Here, the duration of temporary recording is a duration over whichtemporary recording of a moving image is continued, and for example,when a moving image for three seconds before and after shooting of astill image is to be recorded, the duration of temporary recording isset to three seconds. The number of moving image frames or the like maybe set as similar information. Moreover, as the duration of temporaryrecording, a time period determined in advance with respect to eachrecoding mode may be read out from the nonvolatile memory and set, or atime period selected or specified through user operation may be set. TheSW2 flag is a flag for maintaining an ON state from when the releaseswitch (SW2) 64 is turned on to when a sequence of recording processesis completed. The SW2 flag is set to OFF at default.

In the step S302, the system control circuit 50 judges whether or notthe release switch (SW1) 62 has been turned on by, for example, pressingthe release button 61 halfway down, and when the system control circuit50 judges that the release switch (SW1) 62 has been turned on, theprocess proceeds to step S303. When the system control circuit 50 judgesthat the release switch (SW1) 62 is off, the process proceeds to stepS309.

It should be noted that immediately after the initialization in the stepS301, it is assumed that the release button 61 has not been pressedhalfway down in most cases, and in such cases, the process proceeds tothe step S309, in which a moving image before shooting of a still imageis shot and temporarily recorded in the memory 32. On the other hand,when not immediately after the initialization in the step S301, there isa possibility that a still image was shot in step S305, to be describedlater, but in either case, when the release switch (SW1) 62 is off, theprocess proceeds to the step S309, to be described later, in which amoving image is shot and temporarily recorded in the memory 32.

In the step S303, the system control circuit 50 carries out a shootingpreparation process required to shoot a still image and a moving image,and the process proceeds to step S304. Specifically, the system controlcircuit 50 controls the shutter 101, the taking lenses 103, the imagepickup unit 22, and the image processing circuit 24 to carry out an AEprocess, an AF process, an EF process, an AW process, and so on.

In the step S304, the system control circuit 50 judges whether or notthe release switch (SW2) 64 has been turned on by, for example, pressingthe release button 61 all the way down, and when the system controlcircuit 50 judges that the release switch (SW2) 64 has been turned on,the process proceeds to step S305. When the system control circuit 50judges that the release switch (SW2) 64 is off, the process proceeds tothe step S309.

In the step S305, the system control circuit 50 shoots a still imageusing the image pickup unit 22, and the process proceeds to step S306.Specifically, the system control circuit 50 performs light exposure(image pickup) at an f-number, a shutter speed, and a sensitivity, whichhave been set so as to shoot a still image, using the image pickup unit22 and converts an image signal obtained as a result into still imagedata using the A/D converter 23.

It should be noted that, in shooting of a still image, the number ofpixels is larger than that in one frame of a moving image, depending ona set number of pixels to be recorded, and in many cases, the processingtime is longer than that of a shooting process for one frame of a movingimage due to an f-number, a shutter speed, a sensitivity, and imageprocessing.

In the step S306, the system control circuit 50 subjects still imagedata output from the A/D converter 23 to a variety of processing usingthe image processing circuit 24. The system control circuit 50 thentemporarily records the still image data (image data for a still imagefile) and determination information for the still image data in thememory 32, and the process proceeds to step S307. Here, thedetermination information is, for example, a time at which a still imagewas shot, and obtained from the system timer 53. After carrying out aresizing process to resize the still image data to the same size as amoving image recording size, the system control circuit 50 temporarilyrecords moving image frame data at a time at which an instruction toshoot a moving image was issued as determination information in thememory 32.

It should be noted that in the resizing process, when there is adifference in aspect ratio between a moving image and a still image, atrimming process as well is carried out. The determination information,which is temporarily recorded in the memory 32, is for a plurality offrames in a time period to shooting of a moving image in the step S309,to be described later.

In the step 307, the system control circuit 50 turns on the SW2 flag,and the process proceeds to step S308. Here, as described earlier, theSW2 flag is kept on from when the release switch (SW2) 64 is turned onto when a sequential recording process is completed. The SW2 flag beingon indicates that a predetermined time period or less has elapsed aftera last still image was shot. On the other hand, the SW2 flag being offindicates that no still image has been shot since starting of thedigital camera 100, or even when a still image has been shot sincestarting of the digital camera 100, a time period longer than thepredetermined time period has elapsed.

In the step 308, the system control circuit 50 resets a duration oftemporary recording, and the process proceeds to step S313. For example,when a moving image for three seconds before and after shooting of astill image is to be recorded, the duration of temporary recording isset to three seconds. As a result, elapsed time is measured to determinewhether or not a predetermined time period has elapsed since shooting ofthe still image in the step S305.

On the other hand, in the step 309, the system control circuit 50 shootsa moving image using the image pickup unit 22 and converts an imagesignal obtained from the image pickup unit 22 into moving image framedata using the A/D converter 23, followed by the process proceeding tostep S310. The conversion process here is to obtain frame images of themoving image for a moving image file and carried out based on an aspectratio for a moving image, exposure, the number of pixels, and imageprocessing conditions.

In the step 310, the system control circuit 50 subjects the moving imageframe data obtained in the step S309 to a variety of processing usingthe image processing circuit 24 and temporarily records the moving imageframe data and determination information for the moving image frame datain the memory 32, followed by the process proceeding to step S311.

Through the processes in the steps S309 and S310, frame data on a movingimage is accumulated in the memory 32 over a time period for which nostill image is shot. It should be noted that it is only necessary toobtain frame data on a moving image for a predetermined time periodbefore and after shooting of a still image, and hence frame data thatprecedes frame data of twice as long as the predetermined time period(corresponding to time periods before and after shooting of a stillimage) may be discarded successively.

In the step 311, the system control circuit 50 judges whether or not theSW2 flag is on. When the system control circuit 50 judges that the SW2flag is on, the process proceeds to step S312, and when the systemcontrol circuit 50 judges that the SW2 flag is off, the process proceedsto step S313.

In the step 312, the system control circuit 50 performs subtraction forthe duration of temporary recording, and the process proceeds to thestep S313. A value subtracted here corresponds to one frame of a movingimage according to the duration of temporary recording initialized inthe step S301. For example, when the duration of temporary recording isset to 3 seconds in the step S301, and a frame rate of a moving image tobe shot is 60 FPS, 0.016 second is subtracted from 3 seconds. Theprocess in the step S312 is intended to determine whether or not thepredetermined time period has elapsed since shooting of the still image.

In the step 313, the system control circuit 50 judges whether or not theduration of temporary recording is zero. When the system control circuit50 judges that the duration of temporary recording is zero, the processproceeds to step S314, and when the system control circuit 50 judgesthat the duration of temporary recording is not zero, the processreturns to the step S302, in which the sequence of processes is carriedout again. The duration of temporary recording being zero indicates thatthe predetermined time period has elapsed since shooting of the stillimage.

In the step 314, the system control circuit 50 judges whether therecording mode selected in the step S300 is the recording mode 1 or therecording mode 2. When the selected recording mode is the recording mode1, the process proceeds to step S315, and when the selected recordingmode is the recording mode 2, the process proceeds to step S316. In thestep 315, the system control circuit 50 carries out a process in therecording mode 1 using the image processing circuit 24, and the processproceeds to step S317.

In the step 316, the system control circuit 50 carries out a process inthe recording mode 2 using the image processing circuit 24, and theprocess proceeds to the step S317. It should be noted that the processesin the recording mode 1 and the recording mode 2 will be describedlater. In the step 317, the system control circuit 50 judges whether ornot a terminating instruction has been issued through the operating unit70. When the system control circuit 50 judges that a terminatinginstruction has been issued, the process is terminated, and when thesystem control circuit 50 judges that no terminating instruction hasbeen issued, the process returns to the step S301, in which the sequenceof processes is carried out again. It should be noted that when theprocess returns to the step S301, the SW2 flag is initialized to off.

Referring next to FIGS. 4A and 4B, a detailed description will be givenof the processes in the recording mode 1 and the recording mode 2. FIG.4A is a flowchart showing the procedure of the process in the recordingmode 1 in the step S315 in FIG. 3. FIG. 4B is a flowchart showing theprocedure of the process in the recording mode 2 in the step S316 inFIG. 3.

Referring first to FIG. 4A, a description will be given of the recordingmode 1. As shown in FIG. 4A, a plurality of pieces of still image datatemporarily recorded in the memory 32 in response to a plurality ofshooting instructions from the release switch (SW2) 64 is successivelyselected and processed. First, in step S400, the system control circuit50 selects a piece of unprocessed still image data from among aplurality of pieces of still image data temporarily recorded in thememory 32 in response to a plurality of shooting instructions from therelease switch (SW2) 64, and the process proceeds to step S401.

In the step 401, the system control circuit 50 subjects the still imagedata, which was selected in the step S400, to an encoding process usingthe still image encoding-decoding process unit 203 of the imageprocessing circuit 24 and records the still image data as a still imagefile in the recording medium 200, followed by the process proceeding tostep S402.

In the step 402, the system control circuit 50 selects moving imageframe data, which was shot within a predetermined time period before andafter a time at which the still image data selected in the step S400 wasshot, from moving frame data temporarily recorded in the memory 32, andthe process proceeds to step S403. It should be noted that times atwhich still image data and moving frame data are shot are determined bythe system control unit 50 based on determination informationtemporarily recorded in the memory 32 in the step S306 and the stepS310.

In the step 403, the system control circuit 50 subjects the moving imageframe data, which was selected in the step S402, to an encoding processusing the moving image encoding-decoding process unit 204 of the imageprocessing circuit 24 and records the moving image frame data as part ofa moving image in the recording medium 200, followed by the processproceeding to step S404.

In the step 404, the system control circuit 50 judges whether or not themoving image frame data shot within the predetermined time period beforeand after the time at which the still image data selected in the stepS400 was shot is still remaining. When the system control circuit 50judges that the moving frame data is still remaining, the processreturns to the step S402, in which the system control circuit 50 in turnperforms the sequence of processes again, and when the system controlcircuit 50 judges that the moving frame data is not remaining, theprocess proceeds to the step S405. As a result, the moving frame datafor the time period corresponding to one still image selected in thestep S400 is clipped from the overall moving image frame datatemporarily recorded in the memory 32 and recorded as one moving imagefile.

In the step 405, the system control circuit 50 judges whether or not thestill image frame data temporarily recorded in the memory 32 is stillremaining. Upon judging that the still image data is not remaining, thesystem control circuit 50 terminates the process, and when the systemcontrol circuit 50 judges that the still image data is still remaining,the process returns to the step S400, in which the system controlcircuit 50 in turn performs the sequence of processes again.

As a result of the process in FIG. 4A, for each of a plurality of stillimages that has been shot, one still image and one moving image shotwithin a predetermined period before and after shooting of the stillimage are recorded. Namely, in the recording mode 1, a plurality ofmoving image files comprised of pictures for time periods correspondingto respective ones of a plurality of still image files is generated andrecorded.

Referring next to FIG. 4B, a description will be given of the recordingmode 2. As shown in FIG. 4B, in step S406, the system control circuit 50selects a piece of moving frame data from a plurality of pieces ofmoving frame data temporarily recorded in the memory 32, and the processproceeds to step S407.

In the step 407, the system control circuit 50 subjects the moving imageframe data, which was selected in the step S406, to an encoding processusing the moving image encoding-decoding process unit 204 of the imageprocessing circuit 24. After subjecting the moving image frame data tothe encoding process, the system control circuit 50 records the encodedmoving image frame data as a frame of one moving image file in therecording medium 200, followed by the process proceeding to step S408.

In the step S408, the system control circuit 50 judges whether or notthere is any still image data shot at substantially the same time whenthe moving image frame data selected in the step S406 was shot. To beexact, a still image and a moving image are not shot at the same timesince no moving image can be shot during a time period for which a stillimage is being shot (S305 to S308), but data that can be judged as stillimage data that follows a moving image shot immediately before shootingof a still image is regarded as still image data shot at the same timewhen the moving image was shot. The times at which still image data andmoving frame data are shot are determined by the system control unit 50based on determination information temporarily recorded in the memory 32in the step S306 and the step S310. When the system control circuit 50judges that there is still image data shot at substantially the sametime when the moving image frame data was shot, the process proceeds tostep S409, and when the system control circuit 50 judges that there isno still image data shot at substantially the same time when the movingimage frame data was shot, the process proceeds to step S411.

In the step 409, the system control circuit 50 selects still image datashot at substantially the same time when the moving image frame dataselected in the step S406 was shot, and the process proceeds to stepS410. In the step 410, the system control circuit 50 subjects the stillimage data, which was selected in the step S409, to an encoding processusing the still image encoding-decoding process unit 204 of the imageprocessing circuit 24 and records the encoded still image data as astill image in the recording medium 200, followed by the processproceeding to step S411. In the step S411, the system control circuit 50judges whether or not there is any unprocessed moving image frame data.Upon judging that unprocessed moving image frame data is remaining, thesystem control circuit 50 terminates the process, and when the systemcontrol circuit 50 judges that no unprocessed moving image frame data isremaining, the process returns to the step S406, in which the systemcontrol circuit 50 in turn performs the sequence of processes again.

By repeatedly carrying out the process in FIG. 4B, moving image framedata temporarily recorded in the memory 32 is successively joinedtogether to generate one moving image. Through the processes describedabove, one moving image and a plurality of still images shot within atime period for which the moving image is shot are recorded. Namely, inthe recording mode 2, one moving file comprised of pictures for timeperiods corresponding to all of a plurality of still image files isgenerated and recorded.

Referring next to FIGS. 5A and 5B, a description will be given of anexemplary still image temporarily recording process in the step S306 inFIG. 3.

FIG. 5A is a diagram schematically showing a state of still image datatemporarily recorded in the memory 32. In FIG. 5A, T1 and T2 eachdesignate a time at which a shooting instruction is input, and ST1 andST2 designate still image data temporarily recorded in the memory 32 atthe times T1 and T2, respectively.

FIG. 5B is a diagram schematically showing a state of moving image framedata temporarily recorded in the memory 32. In FIG. 5B, STS1 and STS2designate still image data subjected to a predetermined process such asa resizing process at the times ST1 and ST2 and temporarily recorded asmoving image frames in the memory 32. Here, when a processing timeperiod required in the still image shooting mode is longer than that inthe moving image shooting mode, STS1 may be temporarily recorded asmoving image frames in the memory 32 over a plurality of frames.

MM1 and MM2 designate moving image frame data shot in the period fromthe time T1 to the time T2. MB1 to MB15 designate moving image framedata shot in a predetermined time period before the time T1, and MA1 toMA13 designate moving image frame data shot in a predetermined timeperiod after the time T2.

Referring next to FIGS. 6A to 6C, a description will be given of aprocess in which still image data and moving frame data are recorded inthe recording mode 1 in the step S315 in FIG. 3 and the recording mode 2in the step S316 in FIG. 3. It should be noted that in FIGS. 6A to 6C,states of temporarily recorded data correspond to those in FIGS. 5A and5B, and the same reference symbols as those in FIGS. 5A and 5B are used.

FIG. 6A is a schematic diagram showing how still image data is recordedin the recording mode 1 and the recording mode 2. As shown in FIG. 6A,temporarily recorded still image data is subjected to a predeterminedprocess and recorded as a plurality of still images.

FIG. 6B is a schematic diagram showing how moving images are recorded inthe recording mode 1. In the recording mode 1, for a piece of stillimage data, a moving image for a predetermined time period before andafter shooting of the still image data are recorded. For ST1, movingimage frame data MB1 to MA8 shot in a predetermined time period beforeand after the time T1 (MB1 to MB15, STS1, MM1, MM2, STS2, MA1 to MA8) isrecorded as one moving image. For ST2, moving image frame data MB6 toMB13 shot in a predetermined time period before and after the time T2(MB6 to MB15, STS1, MM1, MM2, STS2, MA1 to MA13) is recorded as onemoving image.

FIG. 6C is a schematic diagram showing how a moving image is recorded inthe recording mode 2. In the recording mode 2, temporarily recordedmoving image frame data is recorded as it is as one moving image.Specifically, one moving image (moving image file) comprised of MB1 toMB15, STS1, MM1, MM2, STS2, MA1 to MA13) is recorded as a moving imagecorresponding to both ST1 and ST2.

As described above, in the present embodiment, a moving image in apredetermined time period before and after first shooting of a stillimage and a moving image in a predetermined time period before and aftersecond shooting of a still image are recorded as one moving image in thememory 32. Thus, a moving image in a predetermined time period beforeand after first shooting of a still image and a moving image in apredetermined time period before and after second shooting of a stillimage are processed using a single encoder. This dispenses with aplurality of moving image encoders for carrying out a plurality ofmoving image encoding processes in parallel and a large-capacity memoryfor temporarily recording moving images shot in overlapping moving imageshooting time periods unlike the conventional arts.

Moreover, in the present embodiment, the digital camera 100, whichshoots moving images for a predetermined time period before and aftershooting a still image, is capable of shooting a plurality of stillimages as instructed within a predetermined time period without the needto have a plurality of moving image encoders and a large-capacitymemory.

It should be noted that although in the embodiment described above, theencoding process is carried out in the step S315 and the step S316 aftertemporary recording of a plurality of pieces of moving image frame datais entirely completed in the processes from the step S302 to the stepS311 in FIG. 3, the encoding process should not necessarily be limitedto this. Temporary recording and encoding of still image data and movingimage frame data may be performed in parallel to the extent possible.

Moreover, although in the embodiment described above, moving images arerecorded for the same time period before and after shooting of a stillimage, recording time periods before and after shooting of a still imageshould not always be the same. For example, moving images for twoseconds before and three seconds after shooting of a still image may berecorded.

Further, although in the above description of the embodiment, the casewhere moving image for a predetermined number of seconds before andafter shooting of a still image are recorded is illustrated, only amoving image for a predetermined number of seconds after shooting of astill image may be recorded without recording a moving image beforeshooting of the still image. Namely, the present invention may beapplied even to a case where an instruction to shoot a second stillimage is issued within a predetermined number of seconds after a firststill image is shot. In this case, when the judgment result is negative(NO) in the step S302 in FIG. 3, it is judged whether or not the SW2flag is on. When it is judged that the SW2 flag is on, the processshould proceed to the step S309, and when it is judged that the SW2 flagis off, the process should proceed to the step S317.

Further, although in the embodiment described above, the image pickupapparatus according to the present invention is applied to a digitalcamera, the image pickup apparatus according to the present inventionmay be applied to smart phones and other mobile terminals as long asthey have a function of shooting a moving image for a predetermined timeperiod at least after shooting a still image.

Specifically, the present invention may be applied to a personalcomputer, a PDA, a cellular phone unit, a mobile image viewer, a printerdevice with a display, a digital photo frame, a music player, a gamemachine, an electronic book reader, and so forth. The present inventionmay also be applied to a case where a shooting instruction is receivedas a command from an external apparatus that is remotely controlled.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2015-119066, filed Jun. 12, 2015 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image pickup apparatus comprising: a receivingunit configured to receive a still image shooting instruction; an imagepickup unit; and a control unit configured to control to record, in arecording medium, a still image picked up by said image pickup unit inresponse to a still image shooting instruction received by saidreceiving unit, and a moving image picked up before lapse of apredetermined time period after the still image shooting instruction,and when said receiving unit has received a second still image shootinginstruction within the predetermined time period since said receivingunit received a first still image shooting instruction, control torecord, in the recording medium, a moving image picked up before lapseof the predetermined time period after the second still image shootinginstruction from a moving image shooting starting timing correspondingto the first still image shooting instruction as a sequence of movingimages.
 2. The image pickup apparatus according to claim 1, wherein whensaid receiving unit has received the second still image shootinginstruction after lapse of the predetermined time period since saidreceiving unit received the first still image shooting instruction, saidcontrol unit controls to record, in the recording medium, a moving imagepicked up before lapse of the predetermined time period after the firststill image shooting instruction from a moving image shooting startingtiming corresponding to the first still image shooting instruction and amoving image picked up before lapse of the predetermined time periodafter the second still image shooting instruction from a moving imageshooting starting timing corresponding to the second still imageshooting instruction separately, not as a sequence of moving image. 3.The image pickup apparatus according to claim 1, wherein when saidreceiving unit receives the second still image shooting instructionwithin the predetermined time period after receiving the first stillimage shooting instruction, said control unit controls to generate amoving image in a time period corresponding to the first still imageshooting instruction and a moving image in a time period correspondingto the second still image shooting instruction from the moving imagepicked up before lapse of the predetermined timer period after thesecond still image shooting instruction from the moving image shootingstarting timing corresponding to the first still image shootinginstruction and record the generated moving images in the recordingmedium.
 4. The image pickup apparatus according to claim 1, wherein whensaid receiving unit receives the second still image shooting instructionwithin the predetermined time period after receiving the first stillimage shooting instruction, said control unit controls to record themoving image picked up before lapse of the predetermined timer poredafter the second still image shooting instruction from the moving imageshooting starting timing corresponding to the first still image shootinginstruction as a single moving image in the recording medium.
 5. Theimage pickup apparatus according to claim 1, wherein when said receivingunit receives the second still image shooting instruction within thepredetermined time period after receiving the first still image shootinginstruction, in a first recording mode, said control unit controls togenerate a moving image for a time period corresponding to the firststill image shooting instruction and a moving image for a time periodcorresponding to the second still image shooting instruction from themoving image picked up before lapse of the predetermined timer poredafter the second still image shooting instruction from the moving imageshooting starting timing corresponding to the first still image shootinginstruction and record the generated moving images in the recordingmedium, and in a second recording mode, said control unit controls torecord the moving image picked up before lapse of the predeterminedtimer pored after the second still image shooting instruction from themoving image shooting starting timing corresponding to the first stillimage shooting instruction as a single moving image in the recordingmedium.
 6. The image pickup apparatus according to claim 1, whereinbased on the determination information for use in determining times atwhich a still image and a moving image are shot, said control unitcontrols to record, in the recording medium, a moving image associatedwith one still image and picked up before lapse of the predeterminedtime period after issuance of a still image shooting instruction.
 7. Theimage pickup apparatus according to claim 6, wherein based on thedetermination information, said control unit controls to record, in therecording medium, picked up one moving image in association with aplurality of still images picked up within a time period for which themoving image was picked up.
 8. A control method for an image pickupapparatus comprising having an image pickup unit, comprising: areceiving step of receiving a still image shooting instruction; acontrol step of, when a second still image shooting instruction has beenreceived after a predetermined time period since a first still imageshooting instruction was received in said receiving step, providingcontrol to record, in a recording medium, a still image picked up by theimage pickup unit in response to the first still image shootinginstruction, a still image picked up by the image pickup unit inresponse to the second still image shooting instruction, a first movingimage picked up by the image pickup unit before lapse of thepredetermined time period after the first still image shootinginstruction, and a second moving image picked up by the image pickupunit before lapse of the predetermined time period after the secondstill image shooting instruction; and a control step of, when a secondstill image shooting instruction has been received within thepredetermined time period since a first still image shooting instructionwas received in said receiving step, providing control to record, in therecording medium, a still image picked up by the image pickup unit inresponse to a first still image shooting instruction, a still imagepicked up by the image pickup unit in response to a second still imageshooting instruction, and a moving image based on a moving image pickedup by the image pickup unit before lapse of the predetermined timeperiod after the second still image shooting instruction from a movingimage shooting starting timing corresponding to the first still imageshooting instruction as a sequence of moving images.
 9. A non-transitorycomputer-readable storage medium storing a program for causing acomputer to execute a control method for an image pickup apparatuscomprising having an image pickup unit, the control method for the imagepickup apparatus comprising: a receiving step of receiving a still imageshooting instruction; a control step of, when a second still imageshooting instruction has been received after a predetermined time periodsince a first still image shooting instruction was received in saidreceiving step, providing control to record, in a recording medium, astill image picked up by the image pickup unit in response to the firststill image shooting instruction, a still image picked up by the imagepickup unit in response to the second still image shooting instruction,a first moving image picked up by the image pickup unit before lapse ofthe predetermined time period after the first still image shootinginstruction, and a second moving image picked up by the image pickupunit before lapse of the predetermined time period after the secondstill image shooting instruction; and a control step of, when a secondstill image shooting instruction has been received within thepredetermined time period since a first still image shooting instructionwas received in the receiving step, providing control to record, in therecording medium, a still image picked up by the image pickup unit inresponse to a first still image shooting instruction, a still imagepicked up by the image pickup unit in response to a second still imageshooting instruction, and a moving image based on a moving image pickedup by the image pickup unit before lapse of the predetermined timeperiod after the second still image shooting instruction from a movingimage shooting starting timing corresponding to the first still imageshooting instruction as a sequence of moving images.